BEST AVAILABLE COPY 




QfiSce 



I 



INVESTOR IN PEOPLE 



The Patent Office 
Concept House 
Cardiff Road 
Newport 
South Wales 
NPIO 8QQ 



I, the undersigned, being an officer duly authorised in accordance with Section 74(1) and (4) 
of the Deregulation & Contracting Out Act 1994, to sign and issue certificates on behalf of the 
Comptroller-General, hereby certify that annexed hereto is a true copy of the documents as 
originally filed in connection with the patent application identified therein. 



In accordance with the Patents (Companies Re-registration) Rules 1982, if a company named 
in this certificate and any accompanying documents has re-registered under the Companies Act 
1980 with the same name as that with which it was registered immediately before re- 
registration save for the substitution as, or inclusion as, the last part of the name of the words 
"public limited company" or their equivalents in Welsh, references to the name of the company 
in this certificate and any accompanying documents shall be treated as references to the name 
with which it is so re-registered. 



In accordance with the rules, the words "public limited company" may be replaced by p.Lc, 
pJcJP>L.C. or PLC. 



istration under the Companies Act does not constitute a new legal entity but merely 
the company to certain additional company law rules. 




Signed 

Dated 23 December 2004 



An Kxecutive A eencv of thft DeDartment of Trade and Industrv 



■ 



P-*^nts Form 1/77 



Patents Act 1977 
(KulelG) 



Request for graiit of a patent 

(See tbe notes on ibe back ofUus form. You can also get an 
expIaaaimyieaOet Erom tbe Patent OSBce to befy yon GU in 
tbisfom) 




The Patent Office 

Cardiff Road 
Newport 
South Wales 
NP9 IRH 



1. Your reference 



P36291GB/GVR 



2. Patent application number 

(Ihe Patent Office wiilMin this part) 



0328420.5 



3. Full name, address and postcode of the or of 
each applicant (underline all surnames) 

Patents ADP number Of you know it) 

If the applicant is a corporate body, give the 
country/state of its incorporation 



Rand Ideas Limited 
1 15A Cleveland Street 
London WIT 6PU 



United Kingdom 



4. Htle of the invention 
Modelling iSystem 



5. Name of your agent (if you have one) 

"Address for service" in the United Kingdom 
to vsrhich all correspondence should be sent 
(including the postcode) 



Kilbum & Strode 
20 Red Lion Street 
London 
WC1R4PJ 



Patents ADP number (ifyouJcnowit) 



125001 



6. If you are declaring priority from one or more 
earlier patent applications, give the coimtry 
and the date of filing of the or of each of these 
earlier applications and (ifyouJaiowit) the or 
each application number 



Country Priority application number 

(ifyouiaiowit) 



Date of filing 
(day /month /year) 



If this application is divided or otherwise 
derived from an earlier UK application, 
give the number and the filing date of 
the earlier application 



Number of earlier application 



Date of filing 
(day/mtmth /year) 



Is a statement of inventorship and of ri^t 
to grant of a patent required in support of 
this request? (Answer 'Yes' iJ: 

a) any applicant named in part 3 is not an inventor, or 

b) there is an inventor who is not named as an 
applicant or 

c) any named applicant is a corporate body 
See note (d)) 



YES 



Patents Fonn 1/77 



P-"<5nts Form 1/77 

9. Enter the number of sheets for any of the 
following items you are filing with this form. 
Do not cotmt copies of the same document 

Continuation sheets of this form 
Description 

Claim 
Abstract 




10 

3 




Drawing^? 22 



10. If you are also filing any of the following, 
state how many against each item. 

Priority documents 

Translations of priority documents 

Statement of inventorship and right 
to grant of a patent (Patents Form 7/77) 

Request for preliminary examination 
and search (Patents Form 9/77) 

Request for substantive examination 
(Patents Form 10/77) 

Any other documents 
(please spedfy) 



11. 



lAVe request the grant of a patent on the basis of this application. 
Signature Date g 12.03 



12. Name and daytime telephone number of 
person to contact in the United Kingdom 



Gwilym V. Roberts 
Tel: 020 7539 4200 



Warning 

After an application for a patent has been filed, the Comptroller of the Patent OfHce will consider whether publication 
or communication of the invention should be prohibited or restricted under Section 22 of the Patents Act 1977, You 
will be informed if it is necessary to prohibit or restrict your invention in this way. Furthermore, if you live in the 
United Kingdom, Section 23 of the Patents Act 1977 stops you from applying for a patent abroad without Urst getting 
written permission Srom the Patent Office unless an application has been filed at least 6 weeks beforehand in the 
United IQngdom for a patent for the same invention and either no direction prohibiting publication or 
communication has been given, or any such direction has been revoked. 

Notes 

aj If you need help to fUl in this form or you have any questions, please contact the Patent Office on 0645 500505. 

b) Write your answers in capital letters using black ink or you may type them. 

c) If there is not enough space for all the relevant details on any part of this form, please continue on a separate 
sheet of paper and write "see continuation sheet" in the relevant part(s). Any continuation sheet should be 
attached to this form. 

d) If you have answered Yes' Patents Form 7/77 will need to be Wed. 

e) Once you have Blled in the form you must remember to sign and date it 

f) For details of the fee and ways to pay please contact the Patent Office. 



Patents Form 1/77 



Modelling System 



The invention relates to a modelling system in particular for providing a three 
dimensional model of a built up area. 

Models of this type are useful for a range of applications including urban 
planning and development 

One well known modelling system is provided under the name "City Grid" 
available from Geodata GMBH, Leoben, Austria. According to this system a 
three dimensional urban model is built from aerial photography and street 
survey data combined v^th a large scale two dimensional geographical map 
data such as a GIS database. In particular a "massing model" approach is 
adopted whereby the height of each building on the two dimensional map 
provides a third coordinate to give a three dimensional model extrapolated 
from the two dimensional map, A library of building types can be used to 
replace the derived buildings and hence provide a more detailed model. 

A further approach is described in Friih & Zakhor, University of California, 
Berkeley, "Constructing 3D City Models by Merging Aerial and Ground 
Views" IEEE Computer Graphics and Applications November/December 2003 
pages 52 to 61, according to which an aerial laser scan of an urban area is 
combined with mobile acquisition of fagade data together with mathematical 
image processing techniques. However the system adopted is imprecise in 
view of the goal of obtaining a photo-reaHstic virtual exploration of the city 
and is restricted to buildings. Further more additional information, such as the 
material from which an element is constructed, and which can alter its visual * 
properties, is not extracted at the time of modelling. Furthermore, the 
automated approach described does not permit addition of geometric detail. 
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Various problems arise with existing systems. There are difficulties of the with 
extraction data from the three dimensional model. The accuracy of the model 
derived depends on the accuracy of the underlying geographical data. The 
5 accuracy of the model is also limited by the scope of the library of building 
elements relied upon. Production of known system is generally extremely 
labour intensive and update of the models can be very difficult 

The invention is set out in the claims. 

10 

Embodiments of the invention will now be described by way of example with 
reference to the drawings of which: 

Fig. 1 is a flow diagram showing the steps involved in creating a three 
15 dimensional model and database according to the present invention; 

Fig. 2 shows a sample aerial photograph which can be used to create a three 
dimensional model; 

Fig. 3 shows corresponding map data for correlation with the aerial 
photographs; 

20 Fig. 4 shows a model template for use according to the present invention; 

Fig. 5 is a flow diagram showing the steps involved in combining image data 
according to the invention; 

Fig. 6 shows a three dimensional model derived from the aerial photograph; 
Fig. 7 shows photographic data obtained from a view point; 
25 Fig. 8 shows laser cloud data obtained from a view point; 

Fig. 9 shows correlated photographic and laser cloud image data; 
Fig. 10 shows correlated three dimensional model data; 
Figs 11a to 1 Id show steps involved in determining which view points a 
specific building element can be viewed from; and 



Fig. 12 is a flow diagram showing the steps involved in producing a more 
detailed visual image according to the method. 

In overview, the method described herein uses an aerial photographic plan 
image as a model template for a built up area such as an urban area. 
Geographical data is used' to identify built up area units such as city units 
comprising buildings, for example using postal address as identifier. As a 
result of this the basis for the three dimensional model, forming the model 
template, is aerial data and geographical data is merely used to identify the 
respective city units. The city units can include 'TDuffer zones" including 
additional geographical elements in the environ of the city unit, for example 
trees or letter boxes. As a result all geographical elements are associated with 
an identifiable city unit which in turn can be derived from a standard 
addressing system such as postal address. 

The model template is obtained using a stereo aerial image as a result of which 
a three dimensional model can be derived from the aerial data. The resolution 
and accuracy of the model is improved further by obtaining ground level or 
elevated images using for example photographic or las^r acquisition 
techniques. These ground level or elevated images are correlated such that the 
photographic image can be mapped onto the three dimensional elevational view 
obtained from the laser data. The images are also correlated with the three 
dimensional model obtained firom the aerial image to provide a full 
photographic quality and geographically accurate three dimensional model of 
the built up area. The position of the viewpoints from which die laser or 
photographic images are acquired are stored and represented on the model 
template allowing images from the viewpoint to be accessed through a simple 
link and also allowing simple update of individual city units or parts of the 
three dimensional model. Conversely each city unit can provide a link to all 
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acquired images which show it again using appropriate links. As a result a 
fully integrated database is provided underlying the three dimensional model. 

Referring now to Figs. 1 to 4 and 7, the basic steps involved in creating the 
5 model template and underlying database can be understood. 

At step 100 a plan image of the built up area (Fig. 2) is obtained to provide 
basis for the model template. This is stereoscopic allowing height data to be 
derived as well At step 102, ground or elevated images of the city (Fig. 7) are 

10 obtained from defined viewpoints for example by photographic and/or laser 

acquisition of images. At step 104 city units are identified and their boundaries 
defined from the aerial image. At step 106 the city units are correlated with the 
geographical information (Fig. 3) to provide identifiers such as postal addresses 
for each city unit. At step 108 buffer zones are also assigned to the city unit as 

15 part of the city unit, for example adjacent portions of street and any elements 
such as trees and so forth in that portion to provide the model template 
correlated image shown in Fig. 4 including city unit 402. At step 1 10 each of 
the viewpoints 404 from which ground or elevated images were acquired are 
identified on the model template and at step 1 12 the image data related to each 

20 viewpoint (bearing in mind that multiple images may have been acquired from 
each viewpoint) are Linked to the viewpoint position on the model template. 
For example the image of Hg. 7 is associated with viewpoint 2. At step 1 14 
images firom which each city unit 402 can be seen are also linked to the 
respective city unit providing a fully integrated database underlying the model. 

25 

The manner in which data jfrom various sources is combined can be xmderstood 
with reference to the flow chart shown in Fig. 5 with reference also to Figs. 6 to 
10. At step 500 a three dimensional model template is derived firom the stereo 
aerial photography information (Fig. 6). At step 502 the laser cloud (Fig. 8) 
30 and photographic data (Fig. 7) obtained firom ground level or elevated levels is 
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correlated to obtain fa5ade data (Fig. 9). Li particular the photographic data 
can be mapped onto a three dimensional fa5ade representation obtained from 
the laser cloud. At step 504 the fa5ade date is correlated with the three 
dimensional model template and overlaid onto the three dimensional city units 
5 (Fig. 10) therein such that; at step 506, the full three dimensional model is 
provided also including an mtegrated database of underlymg data of either 
viewpoints and/or city units as discussed above. 

It will be appreciated that various appropriate software techniques and products 
10 can be adopted to implement the method described above as will be apparent to 
the skilled reader, but one advantageous approach is described below. 

The aerial photographic image is obtained by stereo photography and processed 
to obtain the three dimensional geometry using, for example, Stereo Analyst 
15 available from Erdas (www.erdas.com). In order to coincide with existing 
tools such as 3D studio available from Discreet (www.discreet.com) the 3D 
geometry can be created from an inputted triangular stereo aerial photography 
trace. 

20 In order to obtain city unit boundaries and their identifiers, the three 

dimensional geometry, aerial photography and underlying map data are 
overiaid as a result of which the boundaries and postal addresses are obtained. 
Because the aerial image provides the model template the accuracy of the 
database is not limited to the accuracy of the geographical data which serves as 

25 a cross-check only. The geographical data used can be, for example, obtained 
from a GIS database. In addition buffer zones are assigned to each city unit as 
described in more detail above. 



To obtain fafade images, the laser cloud can be obtained using any appropriate 
system, for example Cyra Scanners (www.cyra.com). The photographic data is 
also obtained from one or more viewpoints pear city unit. At least three views 
are preferably obtained, namely left and right of the city unit and central to the • 
unit although even more preferably six views are obtained including elevated 
views as well to avoid dist»rtion with high buildings. Altematively or in 
addition spherical photography can be used to obtain an image of the enture 
building using, for example, spherical cameras available from Spheron 
(www.spheron.com). Yet further the photographic images can be taken from 
adjacent the building and, for example, across the street from the building 
ensuring that details are not lost because they are obscured by intervening items 
when taken from across the street The photographs can be combined and 
assigned to city units using any appropriate tools such as 3D Studio or 
Photoshop available from Adobe (www..adobe.com). However the process can 
be speeded up by layering the three dimensional, photography and map data to 
identify relevant city units. In particular, one city unit will preferably have 
many scans and photographs associated with it, automatically organising the 
data in relation to the city unit means a much faster work flow. 

Facade geometry can be obtained from the "laser cloud" of reference points 
derived from the laser scan. This can be done, for example, by tracing the 
cloud data by identifying base planes and extrusions and mapping onto 
corresponding elements on the photography, for example by identifying city 
units and treating one at a time. Altematively geometry can be traced from the 
photograph and the laser cloud overlaid. The system can embrace, multiple 
viewpoints and use mapping tools capable of various software steps. Those 
software tools and steps include perspective view alignment tools to drape 
photography from multiple viewpoints onto point data, and tools to align three 
dimensional points/planes to image pixels. In addition tools include image 



manipulation tools such as a morph fmiction to create a surface map from two 
or more sources, colour correction between photographs from different Ughting 
conditions and lens distortion correction. Three dimensional trace tools can be 
hnplemented to create faces from cloud data and intuitive cutting and extrusion 
5 tools can be used to build detail from simple surfaces. Photography can be 
automatically mapped to faces produced from the laser cloud data allowing 
"auto bake" textures. As a result simple '^m-wrapped" textures compatible 
with the directX and openGL graphics standards are provided. The data output 
is capable of 3D studio/maya/microstation/autocad/vrml support and provides 
10 support for digital photography including cyhndrical. cubic and sypherical 
panoramic image data as weU as support for laser data from appropriate 
scamiers such as CYRA (ibid). RIEGL (www.giegl.co.at), Zoller & FrohUch 
(www.zofre.de) and MENSI (www.mensi.com). 

15 Implementation of the techniques in detaU will again be supportable by 

appropriate software and can be understood from the flow diagram of Fig. 12. 
At step 1200 liie respective city unit is identified on the model template and 
cross-referenced with an index photograph or laser data. At step 1202 raw 
laser data is loaded; this can be obtained from an auto-reference list against the 

20 identified city unit. At step 1204 the photograph images are loaded once again 
if appropriate from an auto-reference Ust. If spherical imagery is used then this 
is auto-rotated to include liie entire^ity unit. At step 1206 common points or 
planes on a laser data and photographic are isolate. At step 1208 perspective 
ahgnment viewpoints are created as refinement of tiie mark up position, that is 

25 Camera viewpoints recorded on-site have offsets appUed to them to correctiy 
ahgn photography to the data. This is to overcome any inaccuracies in tiie 
recording of on-site camraa positions. 
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At step 1210 the photography is fitted to the cloud data, for example usmg 
known 'Vubber sheet" techniques, projected from the viewpoint. At step 1212 
the most distorted pixels are auto-isolated and these can be replaced with 
imagery from an alternative photographic viewpoint. For example in this or 
other cases imagery from different viewpoints can be mixed where it overlaps 
using for example moiph options or alternatively imagery from one viewpoint 
can be selected over that from other viewpoints. The laser cloud data is 
thereby coloured with information from the photographic pixels. At step 1214 
planes are automatically defined from the fitted photographic unage by 
isolating the coloured laser dots accordmg to user-defined ranges. Some planes, 
wilhm clearly defined shade and colour thresholds (for example representing 
surfaces at right angles to each other, one being Ut by strong hght) can be 
automatically defined, the edges determined and geometry created. Others can 
be 'Tenced'' and isolated by the user to describe less obvious surfaces. At step 
1216 More complex surfaces, and some details, can be created by takmg 2d 
sections through the cloud data and extruding to form planes. Further detail can 
be added by hiding planes other than the surface to be modelled and tracing 
photographic detail or snapping to points within them. At step 1217 edges 
created within the isolated plane (say windows openings in a waU for example) 
are automaticaUy read as "cookie cut" surfaces which can be pushed or puUed 
to produce indentations or extrusions. The resultant surfaces are also 
automaticaUy mapped with relevant photography. In step 1218 surfaces are 
tagged with their material properties and function from pre-defined drop down 
lists such that visual properties are correctly represented. For example 
windows can be tagged as material "glass" defined accordingly as reflective or 
transparent. It wiU be seen that the process is thus significantly accelerated for 
example the automated process of adding photography to the geometry at step 
1217 replaces the lengthy task encountered when using existing tools. 
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.Once the individual units have been fully imaged they are incorporated into die 
model template against the respective city units providing a fuU resolution 
model. 

5 As discussed above the model provides integrated databases by aUowing hnks 
to data accessible via city units or viewpoints such that the underlying image 
data can be accessed from either. The database can incorporate laser scan 
position from the onsite survey including file name, capture time and so forth 
and similarly the data relating to photographic unages taken firom ground level 

10 or elevated positions can be marked up providing a relational data reference to 
record the file names of laser data and photo data as weU as aerial data for each 
city unit. This can be carried out as a preliminary step allowing the detailed 
modelling step described above to be quickly derived from the auto-referenced 
hst carrying details of the photographic, laser and aerial data. 

15 

One particular approach aUowing the database to contain information 
identifying which images show which city units can be understood with 
reference to Figures 1 la to 1 Id. Referring firstiy to Fig. 1 la, a model template 
1 100 includes a plurality of building elements 1 102 defined by boundaries 

20 1 104. A viewpoint from which photographic data is acquired is shown at 1 106. 
Referring to Fig. 1 lb, a plurality of nominal "rays" 1 108 is created emanating 
from the viewpomt 1 106. Any angular resolution can be determined governing 
the number of rays produced and any appropriate radius such as 50 metires can 
be adopted as the maximum ray range beyond which useful image data is not 

25 expected to be acquired. Referring to Fig. 1 Ic each intersection of a given ray 
1 1 10 with a boundary 1 104 is identified and labeUed with the city unit 
• identifier, for example the postal address. Each point of intersection 1 1 12 is 
numbered sequentiaUy in the radially increasing (Erection from the viewpoint 
which is treated as intersection point 1. 



10 



Referring to Fig. 1 Id the extent of each ray extending beyond intersection point 
3 is excised, as is that portion of the ray between intersection point 2 and the 
viewpoint (intersection point 1). The city unit associated with the remaining 
ray segment is therefore visible from the viewpoint and the label attached to the 
intersection point, i.e. the city unit address, is recorded against the viewpoint 
position 1 106. Any dupHcates are merged and as a result it is possible to 
record against a viewpoint each city unit which is visible from it. Conversely 
each city unit may carry a hst of all viewpoints from which it can be seen. 

The invention can be implemented in any appropriate software or hardware or 
firmware and the underlying database stored in any appropriate form such as a 
relational database, HTML and so forth. Individual components can be 
juxtaposed, interchanged or used independently as appropriate. The method 
described can be adopted in relation to any geographical entity for example any 
built up area including urban, suburban, country, agricultural and industrial 
areas as appropriate. 
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Claims 
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1 . A method of producing a three dimensional model of a built up area 
comprising obtaining a plan image of a buHt up area and processing die 
plan image to provide a model template of the built up area by identifying 
boundaries defining built up area units. 



2. A method as claimed in claim 1 funiier comprising correlating the model 
template with a geographical database representing the built up area to 

10 assign identifiers from tiie geographical database to built up area units on 

the model template. 

3. A method as claimed in claim 1 or clahn 2 further comprising obtaining 
image data of the built up area from at least one viewpoint in the built up 

15 area. 

4. A method as claimed in claim 3 in which die image data is at least one of 
laser image scan data and photogr^hic image data. 

20 5. A method as claimed in claim 3 or claim 4 in which the hnage data is 

correlated with the model template to identify built up area unit boundaries. 

6. A method as claimed in any of claims 3 to 5 in which image data showmg a 
built up area unit is linked to the built up area unit on the model template. 



A method as claimed in any of claims 3 to 6 further comprising identifying 
the viewpoint on the model template and Unking image data acquired from 
the viewpoint therewith. 



one 
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8. A method as claimed in claim 7 further comprising tracing at least 
nominal ray from a viewpoint and identifying a built up area unit 
intersected by the ray as visible from the viewpoint. 

9. A method as claimed in any preceding claim in which the built up area unit 
comprises an identifiable geogr^hic element. 

10. A method as claimed in claim 9 in which the built up area unit is 
identifiable by a postal address. 

1 1 . A method as claimed in claim 10 in which the bmlt up area unit further 
comprises geographical elements in an environ associated with the postal 
address. 

12. A method as claimed in any preceding claim in which the plan image is a 
photographic plan image. 



area 



13. A method of producing a three dimensional model of a built up 

comprising obtaining a plan image of the built up area, processing the plan 
image to provide a model template and correlating the plan image with a 
geographical database to assign identifiers to geographical elements on the 
model template. 
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14. A method of producing a three dimensional model of a built up 

comprising providing a model template and priDcessing the model template 
to identify boundaries defining built up area units, in which the built up ^ea 
units include an addressable geographical element and geographical 
elements in the environ thereof. 
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15. A metliod of producing a built up area database comprising providing a 
model template, acquiring image data from at least one viewpoint in the 
built up area, identifying the viewpomt on the model template and 
providing a link from the viewpoint on the model template to the assocated 

5 image data acquired therefrom. 

16. A method of producing a three dimensional model of a built up area 
comprising obtaining photographic image data and laser scan unage data of 
a built up area unit and correlating the photographic image data and laser 

10 scan image data to provide a three dimensional fa9ade unage for the built 

up area unit. 

17. A method as claimed m claim 16 in which the photographic image data is 
spherical photographical image data. 

15 

18. A computer program comprising a set of instructions configured to 
implement the method of any of claims 1 to 17. 

19. A computer readable medium storing a computer program as claimed in 
20 claim 18. 



20. A computer configured to operate under the instructions of a computer 
program as claimed in claim 18. 
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